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DETAILED ACTION 

Information Disclosure Statement 

1 . The Examiner has considered the references listed in the Information Disclosure 
Statement dated January 23, 2004. A copy of the Information Disclosure Statement is 
attached to this office action. 

Claim Objections 

Claim 30 is objected to because of the following informalities: the phrase "SQL 
criteria for locating for spoken words" should be replaced with -SQL criteria for locating 
spoken words-. Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

2. Claims 1, 2, 4, 8-11, 16, 17, 19, and 23-26 are rejected under 35 U.S.C. 102(a) 
as being anticipated by Clements et al. ("Phonetically Searching Applied to On-line 
Distance Learning Modules," IEEE DSP Workshop, October 13-16, 2002), hereinafter 
referred to as Clements. 
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Regarding claims 1 and 16, Clements teaches a phonetic searching technique that 
includes the following steps: 

• defining a phrase to use for searching (p. 2, col. 2, 1J2); 

• defining a minimum confidence level for searching (p. 3, col. 1 , H's 1 and 2, 
ignore results below 90% confidence) ; 

• searching a set of audio segment for said phrase (p. 2, § High-Speed Phonetic 
Searching); 

• producing a set of results of all occurrences of the phrase within the audio 
segments and the confidence that a given occurrence is a match for the search phrase 
(p. 3, col. 1, H's 1 and 2, most likely candidates listed first). 

Regarding claims 2 and 17, Clements teach everything claimed, as applied above (see 
claims 1 and 16, respectively). In addition, Clements teaches: 

• said step of defining includes defining a plurality of phrases (p. 2, col. 2, U 2, 
words or phrases), 

• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speech Phonetic Searching), 

• said step of producing includes producing a set of results of all occurrences of 
the plurality of phrases identified in a specified sequential order within the audio 
segments with said minimum confidence that a given occurrence within said audio 
segments is a match for a corresponding one of said plurality of search phrases (p. 2, 
1f4 through p. 3, fl2). 
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Regarding claims 4 and 19, Clements teach everything claimed, as applied above (see 
claims 1 and 16). In addition, Clements teaches the following: 

• said step of defining includes defining a plurality of phrases (p. 2, col. 2, 1|2), 

• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speech Phonetic Searching), 

• said step of producing includes producing a set of results of all occurrences of 
the plurality of phrases identified in a specified temporal relationship within the audio 
segments with said minimum confidence that a given occurrence within said audio 
segments is a match for a corresponding one of said plurality of search phrases (p. 2, 
col. 2, temporal operators, p. 3, col. 1, H's 1 and 2, § Integration Issues with Infusion). 

Regarding claims 8 and 23, Clements teaches everything claimed, as applied above 
(see claims 1 and 16). In addition Clements teaches: 

• said step of defining includes defining a plurality of phrases (p. 2, col. 2, 1J2),, 

• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speed Phonetic Searching), 

• and said step of producing includes producing a set of results of all occurrences 
of the plurality of phrases identified in a specified temporal relationship within the audio 
segments with said minimum confidence that a given occurrence within said audio 
segments is a match for a corresponding one of said plurality of search phrases (p. 2, 
col. 2, temporal operators, p. 3, col. 1, Confidence__Level, § Integration Issues with 
Infusion). 
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Regarding claims 9 and 24, Clements teaches everything claimed, as applied above 
(see claims 8 and 23). In addition, Clements teaches "said temporal relationship is with 
respect to said phrases" (p. 2, col. 2, representing two phrases spoken within 5 seconds 
of each other). 

Regarding claims 1 1 and 26, Clements teaches everything claimed, as applied above 
(see claims 1 and 16). Clements also teaches "the step of identifying said set of audio 
segments" (§ High-Speed Phonetic Searching, ffs 1 and 2, process the input speech 
producing the phonetic search track), integration Issues with Infusion). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 3, 5, 6, 18, 20 and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Clements in view of Frakes et al. ("Information Retrieval, Data 
Structures & Algorithms," Prentice Hall, 1992), hereinafter referred to as Frakes. 

Regarding claims 3 and 18, Clements teach everything claimed, as applied above (see 
claims 1 and 16). In addition, Clements teaches: 

• said step of defining includes defining a plurality of phrases (p. 2, col. 2, 1J2), 
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• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speed Phonetic Searching), 

• and said step of producing includes producing a set of results of all audio 
segments including (i) at least one occurrence of a selected required one of the plurality 
of phrases (p. 3, col. 1 , H's 1 and 2). 

Clements teaches the use of Boolean operations for queries (abstract), but 
Clements does not specifically teach "(ii) non-occurrences of at least one selected 
forbidden one of said plurality of phrases to be excluded from within the audio 
segments, said occurrence and non-occurrence determined with respect to said 
minimum confidence that a given occurrence within said audio segments is a match for 
a corresponding one of said plurality of search phrases." However, the examiner 
contends that this concept was well known in the art, as taught by Frakes. 

In the same field of endeavor, Frakes teaches basic techniques of information 
retrieval. Frakes' teachings include the use of the Boolean expression "not information" 
to find documents that do not contain information (p. 266), which correspond to 
"selected forbidden one of said plurality of phrases." 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements by specifically providing the "not" 
Boolean operator, as taught by Frakes, since it was well-know in the art that such an 
operation is useful during information retrieval tasks (Frakes is a textbook published in 
1 992 with widely known techniques). 
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Regarding claims 5 and 20, Clements teaches everything claimed, as applied above 
(see claims 1 and 16). In addition, Clements teaches the following: 

• said step of defining includes defining a plurality of phrases (p. 2, col. 2, 1J2), 

• said step of searching includes searching said set of audio segments for said 
plurality of phrases (§ High-Speed Phonetic Searching), 

Clements also teaches the use of Boolean operations for queries (abstract) and 
the use of temporal operators (p. 2, col. 2, 1J2), but Clements does not specifically teach 
"said step of producing includes producing a set of results of all audio segments lacking 
occurrences of the plurality of phrases identified in a specified temporal relationship 
within the audio segments with said minimum confidence that a given occurrence within 
said audio segments is a match for a corresponding one of said plurality of search 
phrases." However, the examiner contends that this concept was well known in the art, 
as taught by Frakes. 

In the same field of endeavor, Frakes teaches basic techniques of information 
retrieval. Frakes' teachings include the use of the Boolean expression "not information" 
to find documents that do not contain information (p. 266), which correspond to 
"selected forbidden one of said plurality of phrases." 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements by specifically providing the "not" 
Boolean operator, as taught by Frakes, since it was well-know in the art that such an 
operation is useful during information retrieval tasks (Frakes is a textbook published in 
1992 with widely known techniques). 
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Regarding claims 6 and 21, Clements in view of Frakes teaches everything claimed, as 
applied above (see claims 5 and 20). In addition, Clements teaches "said temporal 
relationship is with respect to said phrases" (p. 2, col. 2, j|2, representing two phrases 
spoken within 5 seconds of each other). 

4. Claims 7 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Clements in view of Frakes and further in view of Li et al. ("MOQL: A Multimedia Object 
Query Language," Third International Workshop on Multimedia Information Systems, 
Como, Italy, Sept. 1996), hereinafter referred to as Li. 

Regarding claims 7 and 22, Clements in view of Frakes teaches everything claimed, as 
applied above (see claims 5 and 20). In addition, Clements teaches the use of temporal 
operators, but Clements does not specifically teach "said temporal relationship is with 
respect to said audio segment." However, the examiner contends that this concept was 
well known in the art, as taught by Li. 

In the same field of endeavor, Li teaches the use of a multimedia object query 
language that includes the use of temporal functions. Li's teachings include a 
discussion of an anchored specification of time (i.e., a the temporal relationship is with 
respect to the segment) (§3.2). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements in view of Frakes , as taught by Li, 
since this type of temporal operator supports typical multimedia queries (§3.2). 
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5. Claims 10 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Clements in view in view of Li. 

Regarding claims 10 and 25, Clements teaches everything claimed, as applied above 
(see claims 8 and 23). In addition, Clements teaches the use of temporal operators, but 
Clements does not specifically teach "said temporal relationship is with respect to said 
audio segment." However, the examiner contends that this concept was well known in 
the art, as taught by Li. 

In the same field of endeavor, Li teaches the use of a multimedia object query 
language that includes the use of temporal functions. Li's teachings include a 
discussion of an anchored specification of time (i.e., a the temporal relationship is with 
respect to the segment) (§3.2). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements, as taught by Li, since this type of 
temporal operator supports typical multimedia queries (§3.2). 

6. Claims 12, 13, 14, 27 and 28 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Clements in view of Glowny et al. (U.S. Patent Application 
Publication 2001/0040942 A1), hereinafter referred to as Glowny. 

Regarding claim 12, Clements teaches everything claimed, as applied above (see 
claims 11). But Clements does not specifically teach "said step of identifying is 
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responsive to intrinsic data." However, the examiner contends that this concept was 
well known in the art, as taught by Glowny. 

In the same field of endeavor, Glowny discloses a method for recording and 
storing telephone call information. Glowny also teaches the use of additional data (i.e., 
intrinsic data) along with the voice recordings (H 0003). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements by specifically providing additional 
data along with recording, as taught by Glowny, since addition information associated 
with a voice file can aid in searching fl| 0003). 

Regarding claims 13 and 27, Clements teaches everything claimed, as applied above 
(see claims 1 1 and 26), but Clements does not specifically teach "said step of 
identifying is response to CTI data." However, the examiner contends that this concept 
was well known in the art, as taught by Glowny. 

In the same field of endeavor, discloses a method for recording and storing 
telephone call information. Glowny also teaches the use of additional CTI data along 
with the voice recordings ffl 0003). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements by specifically providing additional 
data along with recording, as taught by Glowny, since addition information associated 
with a voice file can aid in searching flj 0003). 
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Regarding claims 14 and 28, Clements in view of Glowny teaches everything claimed, 
as applied above (see claims 13 and 27). But Clements does not specifically teach 
"said CTI data selected from the set consisting of (i) called number (DNIS) and, calling 
number (AN1), and (iii) Agent ID." However, the examiner contends that this concept 
was well known in the art, as taught by Glowny. 

Glowny further discloses the use of telephone number, caller ID and agent ID 
number fl| 0003). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Clements by specifically providing additional 
data along with recording, as taught by Glowny, since addition information associated 
with a voice file can aid in searching (fl 0003). 

7. Claims 15, 29 and 30 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Glowny in view of Clements. 

Regarding claim 15, Glowny teaches a method for recording and storing telephone call 
information. Glowny's method includes the following steps: 

• connecting a plurality of calls to at least one customer service representative 
(Fig. 1, items 130 and 160, Fig. 2, items 100, 230, ffs 0035 through 0037); 

• recording audio segments from each of said plurality of calls (Fig. 1 , item 145, 
Fig. 2, item 180, 0037). In addition, Glown teaches search and retrieval of recoded 
information flj's 0003 and 0009), but Glowny does not specifically teach the following 
steps: 
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a) defining a phrase to use for searching; 

b) defining a minimum confidence level for searching; 

c) searching said set of audio segment for said phrase; and 

d) producing a set of results of all occurrences of the phrase within the audio 
segments and the confidence that a given occurrence is a match for the search 
phrase. 

However, the examiner contends that steps a) through d) were well known in the art, as 
taught by Clements. 

In the same field of endeavor, Clements teaches a phonetic searching technique 
that includes steps a) through d), see rejection of claim 1 , above. 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Glowny by specifically providing the steps a) 
through d), as taught by Clements, since such an approach is an efficient and high- 
speed technique for locating audio information (Clements, p. 1, Introduction lf1, and p. 
2, High-Speed Phonetic Searching, 1|1). 

Regarding claim 29, Glowny discloses a contact center that includes the following: 

• a switch configured to connect each of a plurality of calls to a customer service 
representative workstation (Fig. 2, items 100 and 230, U 0035); 

• a memory connected to said switch and configured to record audio segments 
from each of said plurality of calls (Fig. 1, item 145, Fig. 2, item 180). 
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Glowny also teaches the use of a workstation (supervisory terminal) that can 
browse (search) the recorded information (Fig. 1, item 160, U 0047), but Glowny does 
not specifically teach the following features: 

a) a supervisory terminal configured to define a phrase to use for searching and a 
minimum confidence level for searching 0047; 

b) a search engine connected to said supervisory terminal and to said memory for 
searching said set of audio segment for said phrase; and 

c) a display connected to said search engine and configured to produce a set of 
results of all occurrences of the phrase within the audio segments and the 
confidence that a given occurrence is a match for the search phrase. 

However, the examiner contends that steps a) through c) were well known in the art, as 
taught by Clements. 

In the same field of endeavor, Clements teaches a phonetic searching technique 
that includes the following: a) the ability to define a search phrase and a confidence 
level (p. 2, col. 2, ]J2; p. 3, col. 1 , U's 1 and 2); b) a search engine with necessary 
memory to perform a search (§ High-Speed Phonetic Searching); and c) a necessary 
display for displaying results (p. 3, U's 1 and 2, enumerated results). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Glowny by specifically providing the steps a) 
through c), as taught by Clements, since such an approach is an efficient and high- 
speed technique for locating audio information (Clements, p. 1, Introduction fl1, and p. 
2, High-Speed Phonetic Searching, H1). 
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Regarding claim 30, Glowny discloses a method for recording and storing telephone call 
information includes the following: 

• storing an audio segment in a speech repository (Fig. 1, items 145 and 140, 
110037); 

• storing information regarding the audio segment in a database (Fig. 1, item 155, 
H0037). 

In addition, Glowny teaches the search and retrieval of the information stored in a 
database (H0032) using techniques that are compatible with SQL fl[0225), which 
corresponds to "establishing a search criteria including speech and SQL criteria for 
locating spoken words or phrases... ",but Glowny does not specifically teach the 
following: 

a) ... for locating spoken words or phrases in said audio segment using speech 
recognition technology; 

b) searching said audio segment and said database in accordance with said search 
criteria; 

c) providing a report based on said search. 

However, the examiner contends that steps a) through c) were well known in the art, as 
taught by Clements. 

In the same field of endeavor, Clements teaches a phonetic searching technique 
that includes the following: a) the ability to search for spoken phrases using speech 
recognition techniques (p. 2, col. 2, H's 1 and 2); b) a search engine (§ High-Speed 
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Phonetic Searching); and c) a necessary report for displaying results (p. 3, H's 1 and 2, 
enumerated results). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Glowny by specifically providing the steps a) 
through c), as taught by Clements, since such an approach is an efficient and high- 
speed technique for locating audio information (see p. 1, Introduction 1J1, and p. 2, High- 
Speed Phonetic Searching, 1J1). 



Citation of Pertinent Art 

8. The following prior art made of record but not relied upon is considered pertinent 
to the applicant's disclosure: 

• Kanevsky et al. (U.S. Patent 6,434,520) teach a method for indexing and 
querying audio archives. 

• Petkovic et al. (U.S. Patent 6,1 85,527) teach a method for automatic audio 
content analysis for word spotting, indexing, classification and retrieval. 

• Wilmot et al. (U.S. Patent Application Publication 2002/0147592 A1 ) teach a 
method for searching recorded speech and retrieving relevant segments. 



Conclusion 

Any response to this office action should be mailed to: 

Commissioner of Patents and Trademarks 
P.O. Box 1450 
Alexandria, VA 22313-1450 
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or faxed to: 

(703)872-9314 

Hand-delivered responses should be brought to: 

Crystal Park II 

2121 Crystal Drive 

Arlington, VA. 

Sixth Floor (Receptionist) 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dr. V. Paul Harper whose telephone number is (703) 
305-4197. The examiner can normally be reached on Monday through Friday from 8:00 
a.m. to 4:30 p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil, can be reached on (703) 305-9645. The fax phone 
number for the Technology Center 2600 is (703) 872-9314. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Technology Center 2600 Customer Service office 
whose telephone number is (703) 306-0377. 



VPH/vph 
July 12, 2004 





